On Intra-page and Inter-page Semantic Analysis of Web Pages

To make real Web information more machine processable, this paper presents a new approach to intra-page and inter-page semantic analysis of Web pages. Our approach consists of Web pages structure analysis and semantic clustering for intra-page semantic analysis, and machine learning based link semantic analysis for inter-page analysis. Based on the automatic repetitive patterns discovery in structure level and clustering in semantic level, we explore the intra-page semantic structure of Web pages and extend the processing unit from the whole page to a finer granularity, i.e., semantic information blocks within pages. After observing the various hyperlinks, we synthesize the Web inter-page semantic and define an information organizing oriented hyperlink semantic category. Considering the presentation of the hyperlink carrier and intra-page semantic structure, we propose corresponding feature selection and quantification methods, and then exploit the C4.5 decision-tree method to classify hyperlink semantic type and analyze the inter-page semantic structure. In our experiments, the results suggest that our approch is feasible for machine processing.